#mainTextBox The self-organizing map (SOM), also known as the Kohonen map, is a winner-take-all architecture that quantizes the input space, using a distance metric, into a discrete output space, where neighboring regions in the input space are neighbors in the discrete output space. Here we will apply it to neighborhood clustering of random points along a circle using a Euclidean distance metric.
#subtitleTextBox Noise
#mainTextBox The original data comes from the function component and consists of 8 x-y data pairs along a circle, as shown in the ScatterPlot. We add noise to these data pairs through the Noise component, shown stacked on the function component. The noise component implements both Gaussian and uniform noise, with control of both the mean and variance. Here it is set to uniform noise of zero mean and 0.1 variance. Run the input for one epoch and observe the effect of the noise on the original eight exemplars.
#subtitleTextBox Self-organizing map
#mainTextBox This is the basic architecture of the SOM. It consists of a line Kohonen synapse followed by a winner-take-all axon. We have used a transmitter to plot the Kohonen weights, which represent the cluster centers, directly above the input data. In order to make the input data more visible, we will accumulate 160 samples of the x-y pairs (20 per cluster) before updating the scatter plot. Run the simulation for 500 epochs. Repeat several times and observe how often the weights of the Kohonen synapse end up at the cluster centers of the input.
#subtitleTextBox Annealing the neighborhood width
#mainTextBox During the last simulation you may have noticed that the Kohonen weights did not always find the exact cluster centers. This is because the neighborhood width was fixed at 2. SOMs require annealing of the neighborhood width during training to ensure that the map globally represents the input space. To this end, we have added a scheduler to decrease the neighborhood width to zero, and a matrix viewer to monitor it. Repeat the experiment and notice there is almost always perfect convergence.
#subtitleTextBox Varying the number of PEs
#mainTextBox The number of PEs does not have to be the same as the number of clusters. So, in order to demonstrate the importance of annealing and choosing the proper initial neighborhood width, enter a new value for the number of PEs and initial neighborhood width. You will generally find that in order to obtain good results, it is necessary to set the initial neighborhood width to at least one-half the number of PEs.
#subtitleTextBox A more difficult 1-D problem
#mainTextBox One of the appealing properties of Kohonen maps is their ability to "cover" the input space, even where there are no well-defined clusters. Here the input consists of a 16-sample sine wave with additive uniform noise. Run the network and observe that even though the input data density is contiguous, the line of Kohonen weights covers the entire sine wave. Notice also that the input data density is slightly higher at the extremum of the sine wave, and that more Kohonen weights relocate there.
#subtitleTextBox A dimensional mismatch
#mainTextBox Let's see what happens when we try to apply a line Kohonen to a problem that seems better suited to a 2-dimensional Kohonen map. Here we use the noise component to generate random points within a 2-dimensional box. Run the experiment and observe how the Kohonen line "snakes" around trying to cover the input space.
#subtitleTextBox Square Kohonen
#mainTextBox The 2-dimensional uniform noise problem is better suited to a 2-dimensional Kohonen map. The Square Kohonen synapse, used here, treats all neighbors in the horizontal, vertical and diagonal directions as being within a neighborhood of one. Also available is the Diamond Kohonen synapse, which only considers the horizontal and vertical directions for the immediate neighborhood. Enter values for the number of PEs/square side and an initial neighborhood width, and run the training. You will generally find that, in order to get good results, it is necessary to set the initial neighborhood to at least the number of PEs/square side.
#subtitleTextBox A more difficult 2-dimensional problem
#mainTextBox Using only the function and noise components, we have created an input consisting of two overlapping square random regions, but where the input density is higher at positive and negative extremes along the x-axis. Run the network and observe that the Kohonen weights cover the entire input space, while still reflecting the input data density, so that regions of higher density have more Kohonen weights.
#subtitleTextBox Summary
#mainTextBox You have seen how NeuroSolutions implements Kohonen's self-organizing maps, both one and two dimensional, and how easy it is to watch the formation of the maps on a scatter plot. You have also seen the importance of annealing the neighborhood width and how this can be accomplished with the scheduler.
